Field of the Invention
The present invention relates to an apparatus and method for evaluating the total quality of a wireless communication network. The apparatus and method of the present invention is particularly effective for real-time audio quality evaluation of signals transmitted in the wireless communication network.
The cellular industry requires a reliable method and apparatus for determining that the intended radio coverage for a particular system has been established. Conventionally, radio coverage is determined by physically driving to a large number of test locations within the intended coverage area and both objectively and subjectively grading the delivered audio quality at each test location.
Conventional techniques for objectively grading the delivered audio signal rely on traditional static test methods, such as determining the Signal Including Noise And Distortion (SINAD) for each test location. SINAD equals (S+N+D)/(N+D), where S is the power level of a test tone transmitted across the network with a known frequency, N is the noise level, and D is the distortion level. The noise plus distortion level (N+D) is determined by filtering the received signal with a notch filter to substantially remove the frequencies of the test tone, refiltering the signal with a wide-band filter, and then integrating the doubly filtered signal. The power level S is determined by filtering the received signal with a narrow band filter to isolate the test tone frequencies and then integrating the resulting signal. A receiver measures the total power, noise and distortion at each test location, and SINAD, which represents an indication of the relative signal purity received at each test location, is calculated from these measurements,.
SINAD is currently the standard in evaluating analog cellular networks such as the Advanced Mobile Phone Service (AMPS) employed widely in the U.S. and abroad. SINAD also correlates well with subjective Mean Opinion Score (MOS) analysis.
Mean Opinion Score (MOS) analysis requires listeners to subjectively assign a value, ranging from 1 to 5, to the received signal based on their impressions of audio quality. System evaluators sometimes use a training phase before system evaluation in order to "anchor" a group of listeners. If a training phase is not used, system evaluators use test phrases with known MOS levels to normalize listener bias. System evaluators must use a standard set of reference signals to allow comparisons between test sessions.
System evaluators frequently use MOS analysis to evaluate speech coding algorithms. One advantage of MOS analysis is that listeners are free to assign their own meanings of "good" to the processed speech, making the analysis applicable to a wide variety of distortion types. This freedom, however, also creates a disadvantage in that a listener's scale of "goodness" can vary greatly. Thus, MOS analysis is affected by both choice of listeners as well as pre-evaluation instructions. Additionally, system evaluators must pay particular attention to maintaining consistent test conditions, including: the order of presentation of the speech samples; the type of speech samples; the method of presentation; and the environmental listening conditions. Even with carefully controlled conditions, direct comparison of MOS analysis for different systems is difficult. Furthermore, MOS analysis is usually very expensive and requires more than just a few listeners.
To address the difficulties associated with MOS analysis, analysts have developed objective methods for testing analog cellular networks which also correlate well with MOS analysis results. This correlation is attainable because analog cellular systems are fairly independent of particular test signals and signal distortion effects are well known (such as additive signal level dependent white noise when no amplitude saturation occurs). These objective methods employ either a transmitted sinusoidal signal or a transmitted frequency-band limited noise signal which is then analyzed as received by a receiving unit.
Techniques, both subjective and objective, which effectively measure the performance of analog cellular systems have not proven as effective in evaluating digital cellular systems. Digital cellular systems utilize sophisticated speech coding techniques to process human speech which to date cannot adequately be assessed using known analog testing techniques. Speech coding techniques do not contain flat-spectrum quantizing distortion as do analog systems and utilize signal compression to provide data bandwidths higher than those found in analog systems. Although compression allows higher data bandwidths, it introduces types of distortion not found in analog systems and not easily measured using analog techniques.
The dynamic and nonlinear nature of digital compression (as opposed to relatively linear and time-invariant nature of analog cellular systems) introduces distortions affecting audio quality such as: long delays; bit error bursts; speech clipping; speech muting; speech gaps; and repeats of incorrect segments of speech. Moreover, fixture transmission technologies and telecommunication systems such as half-rate and wideband codecs, ATM (Asynchronous Transfer Mode) protocols, and Broadband Integrated Services Digital Networks (B-ISDN) can produce even more distortions of differing magnitudes and types that most likely also cannot readily be handled by known analog test signals and procedures.
Finally, transmission technologies for Universal Personal Telecommunications/Personal Communications Services (UPT/PCS) may require new performance measurements similar to those used for digital cellular systems because their performance is based in part on the quality of speech in the received signal.
Cellular analysts have used SINAD in digital cellular systems, as well as developed variants of SINAD, in an effort to analyze distortions in these digital cellular networks. The developed variants usually include some type of Fourier analysis to indicate frequency, phase, and amplitude distortions in addition to signal and noise measurements. Subsequently, a comparison is made between resultant error signals, which are defined as the difference between the actual received signal and the transmitted signal. However, to the extent known, these variants do not adequately assess the dynamic and nonlinear nature of digital compression described above, nor do these techniques allow real-time evaluation of the network's quality.
Therefore, not only is there a requirement in the wireless industry for new techniques for measuring distortions in digital systems, but also there is a requirement for new techniques which measure distortion objectively without constant human supervision, cheaply, and correlate well with subjectively determined results. In addition, there is a need for an apparatus and method that provides real-time evaluation of signal quality, voice quality, and overall operating quality of the wireless communication network. The overall operating quality may be evaluated by providing an apparatus that can collect call progress statistics such as, access failures, access attempts, dropped calls, etc.